Semantic Composition of Multimodal Actions in Constraint-based Grammars
نویسندگان
چکیده
The use of speech-accompanying hand gestures to depict objects, to structure the discourse or to give directions is ubiquitous in face-to-face interaction. In this paper, we analyse multimodal signals consisting of speech and gesture from the perspective of semantic composition in constraint-based grammars: we elevate standard methods from linguistics to description of multimodal input so as to connect the semantics of the gestural signal to the semantics of the speech signal and to produce an integrated logical form.
منابع مشابه
Semantic Constraint and QoS-Aware Large-Scale Web Service Composition
Service-oriented architecture facilitates the running time of interactions by using business integration on the networks. Currently, web services are considered as the best option to provide Internet services. Due to an increasing number of Web users and the complexity of users’ queries, simple and atomic services are not able to meet the needs of users; and to provide complex services, it requ...
متن کاملAnalysing Speech and Co-Speech Gesture in Constraint-based Grammars
This paper addresses the form-meaning relation of multimodal communicative actions by means of a grammar that combines verbal input with hand gestures. Unlike speech, gesture signals are interpretable only through their semantic relation to the synchronous speech content. This relation serves to resolve the incomplete meaning that is revealed by gestural form alone. We demonstrate that by using...
متن کاملAn Algebra for Semantic Construction in Constraint-based Grammars
We develop a framework for formalizing semantic construction within grammars expressed in typed feature structure logics, including HPSG. The approach provides an alternative to the lambda calculus; it maintains much of the desirable flexibility of unificationbased approaches to composition, while constraining the allowable operations in order to capture basic generalizations and improve mainta...
متن کاملLexicalized Well-Founded Grammars: Learnability and Merging
This paper presents the theoretical foundation of a new type of constraint-based grammars, Lexicalized Well-Founded Grammars, which are adequate for modeling human language and are learnable. These features make the grammars suitable for developing robust and scalable natural language understanding systems. Our grammars capture both syntax and semantics and have two types of constraints at the ...
متن کاملMulti-commodity Multimodal Splittable Logistics Hub Location Problem with Stochastic Demands
This study presents a multimodal hub location problem which has the capability to split commodities by limited-capacity hubs and transportation systems, based on the assumption that demands are stochastic for multi-commodity network flows. In the real world cases, demands are random over the planning horizon and those which are partially fulfilled, are lost. Thus, the present study handles dema...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 2011